Osage County
LLMs as Method Actors: A Model for Prompt Engineering and Architecture
We introduce "Method Actors" as a mental model for guiding LLM prompt engineering and prompt architecture. Under this mental model, LLMs should be thought of as actors; prompts as scripts and cues; and LLM responses as performances. We apply this mental model to the task of improving LLM performance at playing Connections, a New York Times word puzzle game that prior research identified as a challenging benchmark for evaluating LLM reasoning. Our experiments with GPT-4o show that a "Method Actors" approach can significantly improve LLM performance over both a vanilla and "Chain of Thoughts" approach. A vanilla approach solves 27% of Connections puzzles in our dataset and a "Chain of Thoughts" approach solves 41% of puzzles, whereas our strongest "Method Actor" approach solves 86% of puzzles. We also test OpenAI's newest model designed specifically for complex reasoning tasks, o1-preview. When asked to solve a puzzle all at once, o1-preview solves 79% of Connections puzzles in our dataset, and when allowed to build puzzle solutions one guess at a time over multiple API calls, o1-preview solves 100% of the puzzles. Incorporating a "Method Actor" prompt architecture increases the percentage of puzzles that o1-preview solves perfectly from 76% to 87%.
- North America > United States > California > Los Angeles County > Los Angeles (0.14)
- North America > United States > New York (0.04)
- Europe > France (0.04)
- (2 more...)
- Research Report (1.00)
- Workflow (0.71)
- Leisure & Entertainment > Games (1.00)
- Media (0.67)
On Functional Activations in Deep Neural Networks
Nencka, Andrew S., Muftuler, L. Tugan, LaViolette, Peter, Koch, Kevin M.
Background: Deep neural networks have proven to be powerful computational tools for modeling, prediction, and generation. However, the workings of these models have generally been opaque. Recent work has shown that the performance of some models are modulated by overlapping functional networks of connections within the models. Here the techniques of functional neuroimaging are applied to an exemplary large language model to probe its functional structure. Methods: A series of block-designed task-based prompt sequences were generated to probe the Facebook Galactica-125M model. Tasks included prompts relating to political science, medical imaging, paleontology, archeology, pathology, and random strings presented in an off/on/off pattern with prompts about other random topics. For the generation of each output token, all layer output values were saved to create an effective time series. General linear models were fit to the data to identify layer output values which were active with the tasks. Results: Distinct, overlapping networks were identified with each task. Most overlap was observed between medical imaging and pathology networks. These networks were repeatable across repeated performance of related tasks, and correspondence of identified functional networks and activation in tasks not used to define the functional networks was shown to accurately identify the presented task. Conclusion: The techniques of functional neuroimaging can be applied to deep neural networks as a means to probe their workings. Identified functional networks hold the potential for use in model alignment, modulation of model output, and identifying weights to target in fine-tuning.
- Health & Medicine > Therapeutic Area > Neurology (1.00)
- Health & Medicine > Diagnostic Medicine > Imaging (1.00)
'OneDrive 3.0' shows off sharing, Office, AI roadmaps
OneDrive is investing in improved sharing and organizational features for business users, but consumers are being left behind: Microsoft will add media-centric search and AI capabilities, but that won't occur until next year. That will include the ability to search photos through facial recognition, a feature Google debuted about seven years ago. Microsoft executives referred to the updates as "OneDrive 3.0," a salt shaker of granular features being added to Microsoft's cloud storage. For now, the biggest changes are in organization: You'll be able to search for OneDrive files in a people view, as Microsoft has tipped before. OneDrive is also using some of the recent Windows 11 22H2/23H2 updates to File Explorer as a model to place recommended files in a carousel view at the top of the OneDrive page.
A Hierarchical Approach to exploiting Multiple Datasets from TalkBank
TalkBank is an online database that facilitates the sharing of linguistics research data. However, the existing TalkBank's API has limited data filtering and batch processing capabilities. To overcome these limitations, this paper introduces a pipeline framework that employs a hierarchical search approach, enabling efficient complex data selection. This approach involves a quick preliminary screening of relevant corpora that a researcher may need, and then perform an in-depth search for target data based on specific criteria. The identified files are then indexed, providing easier access for future analysis. Furthermore, the paper demonstrates how data from different studies curated with the framework can be integrated by standardizing and cleaning metadata, allowing researchers to extract insights from a large, integrated dataset. While being designed for TalkBank, the framework can also be adapted to process data from other open-science platforms.
- North America > United States > Pennsylvania > Allegheny County > Pittsburgh (0.04)
- North America > United States > Oklahoma > Osage County (0.04)
- North America > United States > New Jersey > Bergen County > Mahwah (0.04)
- (2 more...)
To ChatGPT, or not to ChatGPT: That is the question!
Pegoraro, Alessandro, Kumari, Kavita, Fereidooni, Hossein, Sadeghi, Ahmad-Reza
ChatGPT has become a global sensation. As ChatGPT and other Large Language Models (LLMs) emerge, concerns of misusing them in various ways increase, such as disseminating fake news, plagiarism, manipulating public opinion, cheating, and fraud. Hence, distinguishing AI-generated from human-generated becomes increasingly essential. Researchers have proposed various detection methodologies, ranging from basic binary classifiers to more complex deep-learning models. Some detection techniques rely on statistical characteristics or syntactic patterns, while others incorporate semantic or contextual information to improve accuracy. The primary objective of this study is to provide a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. Additionally, we evaluated other AI-generated text detection tools that do not specifically claim to detect ChatGPT-generated content to assess their performance in detecting ChatGPT-generated content. For our evaluation, we have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains and user-generated responses from popular social networking platforms. The dataset serves as a reference to assess the performance of various techniques in detecting ChatGPT-generated content. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
- North America > United States > Oklahoma > Osage County (0.04)
- Europe > Germany > Hesse > Darmstadt Region > Darmstadt (0.04)
- Health & Medicine > Therapeutic Area (0.94)
- Education (0.93)
Like a Shem that brought Golem to life
A chatbot is a great way to make internet communication more pleasant for both customers and companies. At the beginning of the millennium, people would probably laugh at you for this sentence. The main reason for this change is Natural Language Processing (NLP). It is this branch of artificial intelligence science, that has transformed the clumsy and cumbersome automata into today's clever chatbots, which you can hardly tell from people sometimes. Thanks to NLP, artificial intelligence learns to understand something as complex as human communication.
Patch Reordering: a Novel Way to Achieve Rotation and Translation Invariance in Convolutional Neural Networks
Shen, Xu, Tian, Xinmei, Sun, Shaoyan, Tao, Dacheng
Convolutional Neural Networks (CNNs) have demonstrated state-of-the-art performance on many visual recognition tasks. However, the combination of convolution and pooling operations only shows invariance to small local location changes in meaningful objects in input. Sometimes, such networks are trained using data augmentation to encode this invariance into the parameters, which restricts the capacity of the model to learn the content of these objects. A more efficient use of the parameter budget is to encode rotation or translation invariance into the model architecture, which relieves the model from the need to learn them. To enable the model to focus on learning the content of objects other than their locations, we propose to conduct patch ranking of the feature maps before feeding them into the next layer. When patch ranking is combined with convolution and pooling operations, we obtain consistent representations despite the location of meaningful objects in input. We show that the patch ranking module improves the performance of the CNN on many benchmark tasks, including MNIST digit recognition, large-scale image recognition, and image retrieval. The code is available at https://github.com//jasonustc/caffe-multigpu/tree/TICNN .
- Asia > Japan > Honshū > Chūbu > Ishikawa Prefecture > Kanazawa (0.05)
- Oceania > Australia (0.04)
- North America > United States > Oklahoma > Osage County (0.04)
- Asia > China > Anhui Province > Hefei (0.04)
Augmented Intelligence: A new way forward for utilities to unite artificial intelligence with the human workforce
When artificial intelligence is brought up in conversation, the classic idea of a robot versus a human emerges – somewhat of an us-versus-them mentality – but artificial intelligence works at its best when it – machine learning, natural language processing, and robotics – is viewed as a partnership with the human workforce. Enter augmented intelligence, which sits at the nexus between artificial intelligence and humans, and revolves around technology helping people to complete their work more efficiently and allowing them to focus more on high-value "human-only" type activities. Today's utilities are faced with multiple market disruptions including the proliferation of distributed energy sources, evolving regulatory and policy changes, the increased adoption of energy efficiency products and programs, changing consumer behaviors, and an imperative to modernize their technologies and processes. Faced with these disruptions, utility executives can leverage innovative approaches such as augmented intelligence to position themselves for success. Utilities make investments in new equipment by upgrading existing assets, such as transformers and substations, and performing preventative maintenance -- all with the goal of improving reliability of service.
- Energy > Power Industry > Utilities (0.51)
- Transportation > Ground > Road (0.48)